Deep Multitask Learning for Semantic Dependency Parsing

We present a deep neural architecture that parses sentences into three semantic dependency graph formalisms. By using efficient, nearly arc-factored inference and a bidirectional-LSTM composed with a multi-layer perceptron, our base system is able to significantly improve the state of the art for semantic dependency parsing, without using hand-engineered features or syntax. We then explore two multitask learning approaches---one that shares parameters across formalisms, and one that uses higher-order structures to predict the graphs jointly. We find that both approaches improve performance across formalisms on average, achieving a new state of the art. Our code is open-source and available at https://github.com/Noahs-ARK/NeurboParser.Comment: Proceedings of ACL 201

Entry level employment in Bristol's creative industries sector (ELEBCIS)

The Entry Level Employment in Bristol’s Creative Industries Sector (ELEBCIS) research project took place between July 2012 – March 2013.The project included primary research in the form of interviews with: young people working with, and professionals working for, informal education providers across the Bristol area; professionals working in formal education (at school, in Further Education and in Higher Education Institutions); Creative and Digital Sector Employers and Employees in a range of companies of different sizes located in the city; focus group meetings with young people in formal education settings in different areas of the city, each with comparatively different intakes and traditions of progression for students into both Higher Education and in to employment in these sectors to date. Secondary research took place in the form of a literature review.Together, these approaches have enabled the development of understandings about a wide range of issues connected to current and future-predicted entry level employment opportunities in the creative and digital (and other) sectors in Bristol, and have enabled the ‘reflecting back’ of current practice, identification of particular barriers, and potential opportunities for innovation which may help to address these. The process has also enabled the identification of effective (as well as less effective) skills training and development provision which is supporting young people across Bristol to access employment in these sectors.Through this process, it has been possible to develop a series of recommendations and proposals for Bristol City Council’s Economy, Enterprise and Inclusion Team – who commissioned this work – to consider in future planning and decision-making related to skills training provision for these sectors in the city in the future.In this way, this report may contribute to the extensive planning and development activity that is currently taking place across the city and the wider region in response to the ambitious plans for the Local Enterprise Partnership’s Bristol Temple Quarter Zone (BTQZ), an Enterprise Zone development in the centre of the city, which are rapidly taking shape.Additionally, it contributes to the growing body of literature relevant to intermediary employers in the Creative Industries sector, defining 'Community Media Sector Connector' organisations and highlighting their role in enabling access, and addressing the poor labour force diversity evident in these sectors.Additionally, the report highlights the lack of a shared definition of the terms 'entry level' and 'Crative Industries,' in existing literature.The report presents a series of recommendations and associated areas for future work which may support the achievement in practice of ambitions which ensure that all residents are able to access and benefit from the economic opportunities being developed in the region and crucially that no-one in the city is ‘left behind’ - a key strategic aim for the West of England Local Enterprise Partnership

Neural Motifs: Scene Graph Parsing with Global Context

We investigate the problem of producing structured graph representations of visual scenes. Our work analyzes the role of motifs: regularly appearing substructures in scene graphs. We present new quantitative insights on such repeated structures in the Visual Genome dataset. Our analysis shows that object labels are highly predictive of relation labels but not vice-versa. We also find that there are recurring patterns even in larger subgraphs: more than 50% of graphs contain motifs involving at least two relations. Our analysis motivates a new baseline: given object detections, predict the most frequent relation between object pairs with the given labels, as seen in the training set. This baseline improves on the previous state-of-the-art by an average of 3.6% relative improvement across evaluation settings. We then introduce Stacked Motif Networks, a new architecture designed to capture higher order motifs in scene graphs that further improves over our strong baseline by an average 7.1% relative gain. Our code is available at github.com/rowanz/neural-motifs.Comment: CVPR 2018 camera read

Backpropagating through Structured Argmax using a SPIGOT

We introduce the structured projection of intermediate gradients optimization technique (SPIGOT), a new method for backpropagating through neural networks that include hard-decision structured predictions (e.g., parsing) in intermediate layers. SPIGOT requires no marginal inference, unlike structured attention networks (Kim et al., 2017) and some reinforcement learning-inspired solutions (Yogatama et al., 2017). Like so-called straight-through estimators (Hinton, 2012), SPIGOT defines gradient-like quantities associated with intermediate nondifferentiable operations, allowing backpropagation before and after them; SPIGOT's proxy aims to ensure that, after a parameter update, the intermediate structure will remain well-formed. We experiment on two structured NLP pipelines: syntactic-then-semantic dependency parsing, and semantic parsing followed by sentiment classification. We show that training with SPIGOT leads to a larger improvement on the downstream task than a modularly-trained pipeline, the straight-through estimator, and structured attention, reaching a new state of the art on semantic dependency parsing.Comment: ACL 201